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CHAPTER I 



THE STUDY OF CRITICAL READING 



I ntroduction 



Background of the Problem 

Teachers in the United States have generally achieved a high 
level of success in teaching the mechanics of reading as evidenced 
by the fact that most graduates of American schools have mastered 
the basic reading skills. There is evidence, hcMewer, that a large 
percentage of the literate population are not discriminating readers 
who evaluate what they read. Chase (10) cal led this condition a higher 
level of illiteracy and described the "higher illiterate" as one who 
can absorb and repeat ideas found on the printed page but one who does 
not evaluate the ideas or relate them to the life around him. In a 
society where each person shares the responsibility for decisions in 
government, it is imperative that citizens have the ability to under- 
stand printed matter and to evaluate the ideas presented. Furthermore, 
freedom from censorship has s concomitant requirement of deve<oping 
discrimination in the consumer. Therefore, it appears that some 
attention should be given to instruction in the skills of critical 
reading in today's schools if students are to be adequately prepared 
to serve as fully functioning citizens. 

Goals of school curricula often include a statement of the 
desirability of critical reading skills; however, there is seldom 
a sequential plan for the development of these skills below grade seven. 
Further, empirical research in critical reading has been limited mainly 
to the high school levels. Psychological research has shown that once 
habits are established, it is difficult to change them. Thus, the 
didactic way in which reading is taught in the lower and intermediate 
grades may encourage naive acceptance of anything that appears in print. 
Postponement of instruction in critical reading skills could allow 
non-evaluative acceptance of all types of reading material to become 
so well established that subsequent learning and use of these skills 
would be difficult. 

Even though critical reading has seldom been taught in the ele- 
mentary school grades, there is little reason to justify its neglect. 
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Research in child development indicates that very young children of 
three and above are capable of critical reasoning and that five and 
six-year-old children can use all of their thinking abilities. Ele- 
mentary school reading instruction frequently ignores higher level 
thinking skills of children and makes primary use of memory, recall, 
and interpretive abilities. Perhaps the reasons for the discrepancy 
between that which is theoretically possible and that which is practiced 
are (1) that elementary school teachers believe that children need a 
foundation in the basic reading skills before they can learn to read 
critically, (2) some think that children’s growth in basic reading 
skills may be deterred as a result of time spent on instruction in 
critical reading, and (3) teachers do not have knowledge of the skills, 
techniques, or materials to teach critical reading. Therefore, the 
central purpose of this study was to ascertain whether critical reading 
skills could be taught to elementary school children while normal pro- 
gression in other basic reading skills was maintained. Another purpose 
of the study was the identification of factors related to critical 
reading ,abi 1 i ty. 



Definition of Critical Reading 

Critical reading has been given various interpretations ranging 
from a narrow concept to a comprehensive one. For example, it has 
been restricted by some writers to the simple detection of propaganda 
techniques. Others have defined critical reading as encompassing 
nearly all levels of reading beyond the literal level. Smith (44) 
placed critical reading at the highest level in a hierarchy of reading 
comprehension skills. The hierarchy included (1) literal reading, 
understanding the denotation of words, ideas or sentences in context, 
(2) interpretive reading, obtaining deeper meanings not directly 
stated in the text, and (3) critical reading, evaluating the quality, 
the value, the accuracy, and the truthfulness of what is read. The 
present research encompasses both interpretive and critical reading 
as described by Smith. Robinson (38) developed the following defi- 
nition of critical reading and made explicit some of the conditions 
that she believed necessary for its development: 

Critical reading "is judgment of the veracity, validity, 
or worth of what is read, based on sound criteria or 
standards developed through previous experiences." To 
develop critical readers, "it is essential that skills 
and abilities in reading for complete understanding 
be combined wi th; 

1. an inquiring attitude; 

2. a background to supply knowledge about the topic, 
field, or area to provide standards or criteria 
for evaluation; 
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3. the ability to suspend judgment until the writer's 
message is fully secured; 

4. the ability to follow the organization or logic of 
the presentation, recognizing what is included 
and what is omitted; 

5. awareness of the author's qualifications and in- 
tent; and 

6. recognition of the publisher's commitments." 

While this definition is more specific than others, it still lacks 
a detailed statement of the skills and abilities that are essential 
to the critical reading process. 

Ri/ssell (41) maintained that critical reading was the appli- 
cation of critical thinking to the process of reading. He defined 
critical thinking as a three-factor ability, which includes an atti- 
tude factor of questioning and suspended Judgment, a conative or 
functional factor which involves use of methods of logical inquiry 
and problem solving, and a judgment factor of evaluating in terms of 
some norm or standard or consensus. 

In the initial stages of this study, the investigators used the 
definition of Robinson (38) and the construct described by Russell (41) 
to formulate the description of the desired behavior of a critical 
reader (see Appendix B for the initial and revised list of behaviors). 
As the study progressed the definition was further refined and expanded 
until a comprehensive set of the behaviors of a mature critical reader 
was obtained. This list of skills can be found in Chapter III on 
pages 20 to 22. Briefly, critical reading is defined in this study as 
an analytical, evaluative type of reading in which the reader analyzes 
and judges both the content of what is stated and the effectiveness 
of the manner in which the material is presented. Reading critically 
involves searching for the purposes underlying the author's message 
and making rational judgments about what is read based upon valid 
criteria. Critical reading skills can be applied to argumentative, 
informational or 1 i terary mater ial . 



Statement of Logical Framework 



The Relationship of Critical Reading 
to the Total Reading Process 

Critical reading is a sub-set. of comprehension skills in the 
total framevv'ork of reading. It encompasses a set of skills which can 
be distinguished for the purposes of instruction and measurement but 
are not completely separable from other reading skills. For example, 
some basic achievements in word recognition skills are necessary before 
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the reader can move to even a minimal performance in literal or inter- 
pretive readings Also, some minimal accomplishments are required in 
both word recognition skills and literal reading before the reader 
can be expected to read critically. Evaluative reading, then. Is 
dependent upon the ability to recognize words and to get the idea the 
author intended. The mature reader is continually perceiving words, 
getting the literal meaning, interpreting, applying, and evaluating 
as he reads. Reading could be viewed, then, as a spiraling continuum 
of reading skills, i.e., the skills are interrelated, overlapping, and 
all are essential to the reading process, 

Critical reading does not begin after the author's ideas have 
been grasped, but is an ongoing part of the process of securing mean- 
ing. Evaluation occurs at every stage of reading as the reader selects 
suitable information and rejects the unsuitable, interprets a descriptive 
phrase, recognizes the techniques of persuasion, or analyzes plot 
development. 



The Relationship of Critical 
Reading to Critical Thinking 

Similar to the idea that critical reading cannot be separated 
from the total reading process is the concept that it cannot be sepa- 
rated from critical thinking. Critical reading is the use of critical 
thinking in the evaluation of ideas presented in printed form. A 
comparison of a list of skills used in critical thinking set forth by 
Ennis (13) with the list of skills used by a critical reader shows 
great similarity. According to Ennis, some of the skills involved in 
the act of critical thinking are judging when a line of reasoning is 
logical, if a conclusion necessarily follows, and if a statement 
made by an alleged authority is acceptable. In the present study these 
skills are applied to printed material. 

It is important to emphasize critical thinking in all areas of 
the elementary curriculum, but it particularly needs to be emphasized 
in the teaching of reading. Teachers too frequently have stressed the 
mechanics of reading to the neglect of the thinking aspect. Often the 
inherent rightness of the printed page is assumed, and children develop 
a non-questioning attitude toward their reading. Since the major 
portion of the School day is devoted to reading instruction, many 
opportunities are provided for teaching critical reading. 

The research efforts have been mainly in the area of critical 
thinking. However, that ability in critical thinking automatically 
transfers to an evaluation of the printed page without training has not 
been established. Hence, it appears that research in critical reading 
is warranted. 
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The Nature of the Critical Reader 



The present study was based upon the assumption that children of 
ail ages can learn to read critically. Research in child development 
has substantiated the belief that children can use higher level reason- 
ing processes by age three. Children of this age have shown that they 
can evaluate ideas that are within the realm of their own experience. 

Almy (1), Wann, ^ al. (53) have reported that pre-school children have 
access to all types of thinking behavior and are limited only by their 
background of experiences. Critical reading is the use of critical 
thinking in the act of reading. If young children can think critically 
one could infer that they can be taught to read critically. 

Findings from other research studies have indicated that certain 
characteristics such as intelligence, personality, attitudes, sex, and 
general reading ability may be related to the ability to read critically. 
Glaser (17) found that intelligence, reading ability, and school marks 
are positively related to scores made on critical thinking tests. It 
could be hypothesized that these same factors will be related to scores 
on a test of critical reading ability. Piekarz (36) supported the idea 
that attitudes affected reading at the perceptual level by helping to 
determine the printed words that were actually seen, at the under- 
standing level by coloring, twisting, and distorting ideas, and at the 
retention level by specifying what was remembered and what forgotten. 
Similarly, Watson (54) described selective perception and retention as 
personality factors that Inhibit change. Examining another factor, 
Waetjen (52) described sex differences in learning which may affect the 
level of skill development achieved by boys or girls. The relationship 
of many of these factors to critical reading ability is explored in 
this study. 



The Nature of Teaching 
Critical Reading 

Studies by Rogers (39), and Gray (18) have shown that the ability 
to analyze and evaluate Ideas does not develop naturally as a concomitant 
of chronological age. Rather, critical reading behavior is learned, 
and proper and systematic instruction In the skills that contribute 
to It are necessary. 

Teaching critical reading Involves a number of identifiable 
acts on the part of the teacher. The teacher needs to recognize the 
goals of critical reading and to be familiar with the list of specific 
skills which contribute to critical reading behavior. 

To obtain more ideas about the nature of teaching critical reading, 
a pilot observation study was conducted prior to the developmental and 
experimental phases of this research study. (For procedures, including 
observation Instrument, and findings, see Appendix A.) Observations 
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Indicated the following components of a critical reading lesson; 

(1) establishing a background of information, (2) refining, analyzing 
and applying the information, and (3) evaluating and summarizing in 
an effort to draw valid conclusions or to make appropriate generali- 
zations. The nature of the teachers* questions determined to some 
extent the type of response that was elicited from the pupils. When 
teachers discussed the material read with the children and asked 
questions requiring evaluation, critical responses occurred more 
frequently than if the teacher asked factual questions. 



Purposes of the Study 



The central purpose of this study was to determine if elementary 
school children could be taught to read critically. More specifically, 
the purposes were; 

1. To determine whether or not critical reading skills can be 
taught to elementary school children while normal progression 
in other basic reading skills is maintained. 

2. To determine whether there is a relationship between ability 
to read critically and other characteristics such as general 
reading ability, intelligence, and personality factors. 

3. To determine what kinds of teacher verbal behavior elicit 
crit.lcal responses from children. 

4. To obtain the reactions of teachers to the process of teaching 
critical reading. 

In order to achieve the major goals, several instrumental goals 
were recognized. These goals were; 

1. To refine and verify a list of critical reading skills. 

2. To develop materials for teaching critical reading skills. 

3. To construct a valid and reliable instrument for measuring 
critical reading abl 1 i ty. 

4. To design an observation procedure for recording and classi- 
fying teacher and student verbal behavior. 



Questions to be Answered in the Study 

1. Can critical reading skills be taught to elementary school 
children while normal progression in other basic reading skills 
is maintained? 

a) Will children who receive special instruction In critical 
reading gain significantly more than those who did not 
receive this special instruction? 

b) Will children who receive special instruction In critical 
reading make comparable gains in general reading to 
children who did not receive this instruction? 

c) Is there a significant difference in the ability of 
children of different grade levels to read critically? 

2. Are there relationships between ability to read critically, 
and certain other factors? 

a) Is there a relationship between critical reading ability 
and general reading ability, Intelligence, and personality? 

b) Do students at all intelligence levels gain in critical 
reading abi 1 i ty? 

c) Do boys and girls gain equally in critical reading ability? 

3. What types of teacher verbal behavior elicit critical responses 
from ch i Idren? 

a) Are there significant differences in the type of questions 
that control and experimental teachers ask? 

b) Are there significant differences in the types of questions 
asked at different grade levels? 

c) Is there a change in the types of questions asked during 
different periods in the school year? 

d) Are there significantly more critical responses elicited 
in the experimental than in the control group? 

e) What types of questions elicited the most critical responses 
from pupi Is? 

f) Is there a change in the level of pupil responses during 
different periods in the school year? 

4. What were the teachers' reactions to their attempts to teach 
children to read critically? What problems did they encounter? 



Overview of the Study 



The purpose of developing critical reading abilities in elementary 
school children is to contribute to a citizenry who can intelligently 
evaluate printed materials. In this study critical reading is identified 
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as a part of reading comprehension and is defined as an analytical, 
evaluative type of reading in which the reader analyzes and judges both 
the content of what is stated and the effectiveness of the way it is 
written. Reading critically involves searching for the purpose under- 
lying the message and making rational judgments about what is read 
based upon valid criteria. 

The research was conducted in two major phases: developmental 
and experimental. The developmental phase included (1) the refinement 
and verification of a list of critical reading skills, (2) a pilot 
observation study for identification of appropriate techniques and 
materials for teaching critical reading, (3) preparation of experimental 
materials, and (k) the construction of a measuring instrument. 

In the experimental phase (1) training sessions were provided 
for all teachers in the study, (2) pretests of critical reading and 
general reading were administered to the subjects in the fall, (3) 
experimental lessons were taught during an entire academic year, (4) 
systematic observations were made, and (5) posttests were administered 
In the spring. 

Literature related to the major aspects of this study is reviewed 
in Chapter II and the procedures used are described in Chapter III. 

The final chapters are devoted to the findings, conclusions, and impli- 
cations of the study. 

\ 
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CHAPTER I I 



REVIEW OF THE LITERATURE 



The term ^'critical reading" appears frequently in educational 
literature but the meaning attached to it varies widely. Much of 
this variability in meaning is the result of insufficient research 
evidence regarding both the abilities inherent in the skill of reading 
crit'cally and the factors related to such competency. Research in 
critical reading that is focused on elementary school children is 
especially meager. In this chapter reports of research on both critical 
reading and critical thinking are included. Pertinent theoretical 
articles are cited when they help to clarify purposes, structure, 
or findings in the area of critical reading. 



Research on Critical Thinking Relevant to Critical Reading 



Critical reading is the application of critical thinking skills 
to the reading act. The research on critical thinking is more extensive 
and generally more experimentally sound than the research on critical 
reading . 

As late as 19^2, Ennis (13) noted a deficiency in the comprehen™ 
siveness of the definition of critical thinking. He states that, 

"There has been a lack of careful attention to the concept 'critical 
thinking' and furthermore, no comprehensive, thorough, up-to-date 
treatment of this concept is available." The researcher provides his 
definition of three dimensions of critical thinking; (l) Logical — 
judging the alleged relationships between meaning of words and statements, 
(2) Criterial — Judging ideas presented, and (3) Pragmatic — Judging 
whether the statement is good enough for the purpose held. This defi- 
nition includes many of the concepts included in critical reading. 

Based upon a research review, Durrell and Chambers (12) noted 
the following essentials for research in thinking; a definition of 
terms, construction of measures of thinking abilities, and the design 
and evaluation of materials and methods of teaching thinking. Many 
of the experimental studies reviewed below reveal an attempt to (1) 
define specific thinking skills, (2) develop materials for teaching 
them, and (3) measure achievement with appropriate instruments. Al- 
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though the scope and precision of the studies vary, the findings gen- 
erally indicate that critical thinking skills can be improved through 
direct Instruction. 

One of the most widely-quoted studies using direct instruction 
was implemented by Glaser (17)* The experimenter developed materials 
and techniques to stimulate growth in ability to think critically and 
tested them in four twelfth-grade English classes for ten weeks, using 
a pretest-posttest control group research design. The average gain 
in critical thinking ability on a battery of tests developed by the 
author was significantly greater for the experimental groups than for 
the control groups. 

Anderson ^ aj, (2) conducted an experiment to determine if 
students could learn to abstract, organize information, and draw con- 
clusions — processes which they identified as critical thinking skills. 
Twelve to fifteen matched pairs of seventh-grade experimental classes 
and the same number of tenth-grade experimental classes were tested 
on their ability to solve critical thinking problems through two 
approaches: "doing and telling." The problems based on the "doing" 

method allowed the pupils freedom to study and analyze the content on 
their own while the material in the "telling" problems was more di- 
rective. The differences were negligible between the two experimental 
groups. The experimental groups were also compared with the control 
groups which received no instruction. The control groups did as well 
on abstracting and organizing information as did the experimental 
groups, but were lower on drawing conclusions. Hence, the authors 
conclude that the critical thinking problems made their most important 
contribution in developing skills associated with the drawing of con- 
clusions. 

Seventh graders were the subjects of another research program 
(25) intended to improve the ability of the students "to think logi- 
cally and therefore critically." The intelligence, mental age, reasoning 
ability, language proficiency and initial reading ability of 200 
children were measured and, from the data secured, two paired and 
equated groups of thirty-three children were formed. At the beginning 
of the study the two groups were equal in reasoning ability, but there 
was a significant difference favoring the experimental group at the 
final testing which showed that logic can be taught to upper-grade 
children. Hyram appears to equate logic with critical thinking. 

Although the investigators of this study include logic as an important 
part of the skills of critical thinking and critical reading, they 
also include other skills as well. 

in a study assessing elementary school children's ability to 
master mathematical logic, Suppes and B inford (47) found that the 
achievement level of the upper quart! le of elementary school students 
was 85 to 90 per cent of that achieved by comparable university students. 
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Brownell (8) studied the effects of an instructional reading 
program on the gain scores of two ninth"*grade classes. The mea^rlng 
InstruiTKsnt was the Watson^Glaser Critical Th !nk!nq Appra!saK The 
data collected provisionally support the hypothesis that a twenty- 
eight week program designed to improve reading skills in social studies 
will result in significant total score gains on the critical thinking 
test. There is no conclusive evidence that the significant gains were 
caused by the training in reading alone. Subjects at the upper snd 
lower levels of the intelligence distribution constituted the majority 
of subjects who made more than a median improvement on the Watson- 
Glaser Test . Brownell believes that the Watson-G laser Critical Thin k m fl 
Appraisal may be a measure of reading ability and raises the question 
of whether this test can be used to measure critical thinking apart 
from reading ability. 

Most of the research studies mentioned above used junior and 
senior high school students as subjects while other researchers have 
focused their efforts on children In the elementary school. For ex- 
ample, Grener and Baths (20) used a third-grade class for a limited 
study of critical thinking. The test employed was one developed by 
Baths (37) and a group of teachers from Euclid, Ohio. The teachers 
had two purposes for constructing such an instrument: first, to analyze 

the causes of confusion in a child's thinking, and second, to make 
teachers more aware of the elements which cause the confusion. The 
group was tested in September and again in January. The experiment 
showed that children can perform higher thinking processes to some 

degree. 

Arnold (3) used a problem discussion technique to teach critical 
thinking to fifth and sixth-grade students. He wanted to determine 
if students could make intelligent judgments about the relevancy, 
dependabi lity,, bias in source, and adequacy of data concerning a par- 
ticular proiiiem, question, or conclusion. A control group was given 
no instruction. Arnold's hypothesis that critical thinking can be 
taught was supported. 

Becently Taba (48) directed a study concerned with the thinking 
of elementary'school'chMdren. Although the researcher stated that 
critical think! ng £§r ^ was not being measured, the experimental 
treatment included the teaching of the skills of interpreting, inferring, 
and generalizing. Considerable growth in the transformation of concrete 
thought into formal thought from the second to the sixth grade was 
found. The data seem to imply that training in thinking accelerates 
the pace of thought development. In other words, children can learn 
to make inferences, to generalize, and to make logical assumptions at 
an early age if they receive systematic instruction in thinking skills. 

As a result of his study of the cognitive development of children, 
Piaget (35) identified age levels at which specific thought processes 
appeared. Formal thought, according to his schema, manifests itself 



around age twelve. The developmental sequence theorized by the Swiss 
psychologist is affirmed in Taba's study, but the age placement of the 
thought processes is not. Taba*s study adds to the growing body of 
evidence to support the claim that the occurrence of formal thought 
can be accelerated by instruction. 

There are a number of similarities between Taba's study and the 
research in critical reading reported here. In each there is recog- 
nit ion of the importance of the following elements: (1) systematic 

Instruction, (2) the nature of teachers' questions, and (3) the teachers' 
confidence that children are capable of a higher level of thought than 
they ordinarily exhibit. 

Taba found the beginnings of formal thought processes in grade 
two; other researchers have evidence that children can handle aspects 
of critical thinking even before that time. Smith (43) examined 20,000 
sentences spoken by children of ages two to six and found 325 statements 
which showed evidence of evaluation. Wann (53) shows chat children 
are capable of a wide range of thought between the ages of three and 
five, and that this ability can be influenced positively by knowledgeable 
teachers. A repeated finding from these studies is that instruction 
accelerates the thinking process. 

The scarcity of valid and reliable instruments for measuring 
critical thinking has resulted in the sporadic growth of knowledge 
about this ability in children. This lack is especially noticeable 
at the elementary school level. A review of the literature on critical 
thinking reveals other general inadequacies of research design and 
treatment including the need for (l) developing an adequate definition 
as a basis for the research, and (2) controlling the experiment suffi- 
ciently so that the findings can be generalized beyond the sample. 



Critical Reading in Relation to Other Reading Skills 



Measurement of reading ability has developed from a time when 
one score represented a child's total reading ability to the current 
status where a detailed explication of specific types of reading skills 
is expected. One of the first educators to define specific skills in 
critical reading was DeBoer (11). He said critical reading was an 
active rather than a passive approach to the printed page, the ability 
to distinguish relevant from irrelevant data, and the existence of 
skepticism so that the reader will evaluate the reliability of evidence 
and the soundness of conclusions. More recent and comprehensive defi- 
nitions have been contributed by Robinson, Russell, and Smith (see pages 
2 and 3 of this report). An early differentiation of a specific critical 
reading skill was attempted in Gans' study. In her study of fourth, 
fifth,, #nd sixth graders, Gans (16) identified the ability to select or 
reject relevant and irrelevant materials as a critical reading skill. 
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In her test, subjects were required to select from relevant sources 
in order to solve a problem and to explore a topic. The students were 
provided with the following five types of material: (1) completely 

descriptive factual material as found in textbooks, (2) descriptive 
material not obviously related, (3) encyclopedic content not really 
relevant to the topic, (4) fanciful and imaginative material from 
literature, and (5) material completely irrelevant but written like 
the other materials. Students were unable to see relevancy in remotely 
relevant material; they selected encyclopedic material even though it 
was not relevant; they accepted fanciful material as acceptable sources; 
and they rejected relevant material if it tended to disprove what they 
were trying to prove. Gans concluded that children were not being 
taught to be discriminative about the authenticity of content or the 
relevancy of materials. Although she recognized that critical reference 
reading was made up of several abilities, she believed that general 
reading ability was the most potent factor. 

Sochor (46) still contended in 1959? that for all practical 
purposes, literal and critical reading could not be differentiated 
except on the basis of the reader's purpose. Other researchers have 
attempted to separate them on other bases. In one of these' studies, 
McCullough (32) analyzed reading comprehension tests to determine if 
they were measuring essentially different things. She identified the 
following four types of questions found in measures of reading compre- 
hension: (1) main idea, (2) facts or details, (3) sequence or organi- 

zation, and (4) creative reading which included drawing inferences and 
conclusions, passing judgments, and seeing relationships. When McCullough 
analyzed the scores of three different tests, she found a positive 
relationship among the four types of comprehension questions but the 
degree of the relationship did not justify the idea of testing children 
by one type in order to discover their ability in other types. Individual 
predictions of scores on one type of comprehension from scores on another 
type were impossible. The area that McCullough labeled creative reading 
encompasses specific skills which are considered in this report to be 
a part of critical reading. Her analysis supports the premise that 
critical reading is a distinct ability and should be measured apart from 
other reading skills if competence in critical reading is to be accurately 
assessed . 

Maney (31) and Sochor (45) also questioned the traditional use of 
a single reading test to measure reading ability in all situations as 
well as the assumption that critical reading skills develop as a con- 
comitant of intelligence, maturation and normal school progression. 

Maney examined the relationship between general reading ability and 
critical reading ability of science material. Using a sample of 513 
fifth-grade pupils, she found the correlation between general reading 
ability and critical reading of science to be only .11 when intelligence 
was partial led out. Sochor used the same sample as Maney to investigate 
the relationship between general reading ability and the ability to 
read critically in social studies. Again, the relationship between 
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general reading ability and critical reading ability of social studies 
material was low; in this case it was .23 with intelligence partial led 
out. Sochor concluded that critical reading con^prehens i on in social 
studies appeared to be virtually independent of ‘‘general** reading 
ability when the effects of intelligence were taken into account. 

Further, Shores and Saupe (42) attempted to find out if reading 
Were a general ability or if reading for problem solving in science 
required distinct skills. The authors state that the study skills 
required by problems in science involved both reading and thinking 
critically. They found a correlation of .63 between science reading 
and general reading ability which suggests that the two tests were 
measuring the same factor to some extent. In this study intelligence 
was not partial led out in the statistical analysis. 

Although many researchers have shown that critical reading 
abilities cannot be inferred from a measure of literal reading, few 
are willing to separate the two entirely. In order to read critically, 
one must first be able to comprehend the literal message intended by 
the author. Artley (4) states that although literal comprehension is 
ordinarily conceived as the process of identifying and recalling facts, 
it need not be a non-critical process. Betts (6) supports the idea 
that literal and critical reading are not an "either-or" process. He 
says; 



Assimilative and critical reading are not dichotomous. Instead, 
depth of comprehension is a matter of degree. Reading of the 
predoTiiinantly assimilative type emphasizes the identification 
and recall of facts. Reading of the predominantly critical 
type emphasizes the higher thought processes having to do with 
the selection-rejection of ideas, the relationships between 
ideas, and the organization of information. 

Gray (18) states that literal reading has a critical nature because 
getting the literal meaning necessitates the accurate perception of 
words, the fusion of separate meanings into ideas, grasping the organi- 
zation and relationship of ideas, and a reasonable fluency of perception. 
He adds, "obviously an inquiring attitude and good thinking are required 
at every step in the apprehension of the literal meaning of a passage." 

In summary, there is evidence that skills labeled as critical 
reading skills can be distinguished from those required for literal 
comprehension. Further, it Is generally believed that literal reading 
skills are basic to critical reading performance. 
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Factors Related to Critical Reading Ability 



There has been some experimentation at the elementary school level 
on factors affecting children's ability to read critically. Relevant 
information on these factors can also be gleaned from research at the 
college and high school levels. Factors which educators believe to 
have a potential relationship to critical reading are attitudes, in- 
telligence, general reading ability, background of experience, person- 
ality, perception, and creative thinking abilities. The bulk of the 
research activity has been with the relationship between critical 
reading and factors of attitudes. Intelligence, and general reading. 



Att i tudes 

Attltudlnal factors have been shown to Influence the meaning 
obtained from reading as well as the facts remembered. Thayer {ks) 
used 112 college sophomores as the subjects of a study which investi- 
gated the relationship between readers' ethical and moral values and 
their conceptualization of a fictitious character and his environment. 
The findings indicate that the readers ascribed characteristics they 
valued to the character in the story. In other words, the values of 
the subjects colored their conceptualization and thus provided a pre- 
conceived frame of reference for their reading. 

Five hundred and tw'elve eleventh-grade students were the subjects 
of an experiment by McKlllop (33) who studied the relationship between 
the reader's attitude and certain types of reading responses. Measure- 
ments were taken of general reading ability and attitudes, with atti- 
tudes measured by tests constructed by the researchers. In the experi- 
ment the students read passages concerning Negro-White equality, living 
conditions under communism, and Arab objections to the establishment of 
Israel. The passages corresponded to the topics on the attitude tests. 
Students answered two types of questions; some of the highly structured 
type, and others calling for the making of Inferences, judgments, and 
evaluation. The results of the study showed that the relationship 
between attitude toward a topic and the response to reading passages 
depended in part on the kind of questions asked. If there were a 
"right" answer to the question, the subjects provided this answer and 
did not reveal their a/n feelings. Attitude became a more Important 
factor if there were no correct answer given in the passage and the 
subjects were provided an opportunity for making a judgment. 

Groff (21) found a positive relationship between expressed attitudes 
toward different types of reading material and scores on a critical 
reading test when he studied 305 fifth and sixth-grade children. The 
critical reading test covered three basic areas; sensing organization, 
making Inferences, and drawing conclusions. Relationships between the 
critical reading scores and (l) attitudes expressed toward reading as 
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a school activity and (2) attitudes expressed toward school, classmates, 
and teachers were also explored. Correlations between general reading 
ability and attitudes toward the four content types of reading material 
Were negligible, but the correlations between attitudes toward content 
and critical reading test scores were significant. Another important 
finding of Groff's study was that significant sex differences were 
observed in the interest shown in specific passages. 

As a result of extensive case studies of two students, Piekarz (36) 
stated that attitudes affect reading at the perceptual level by helping 
to determine the printed words that are actually seen, at the under- 
standing level by coloring, twisting, and distorting ideas, and at the 
retention level by specifying what will be remembered and what will be 
forgotten. On the basis of the findings of the preceding studies, it 
appears that attitudinal factors may influence the reader's ability to 
think critically about what he reads. 



Intelligence and General Reading Ability 

In addition to attitudinal factors, researchers have investigated the 
factors of intelligence and general reading ability as they are related 
to the ability to read and think critically, in Glaser's (17) experi- 
ment on critical thinking, he concluded that general intelligence, as 
measured by the Otis Quick- Scoring Mental Ability Test , is different 
•from the abilities measured by the Watson-Glaser Critical Thinking 
Appra isal , However, the tests overlap and some common abilities are 
assessed. Glaser reported that the two factors most clearly related 
to scores on the critical thinking test were found to be intelligence 
and reading ability. The correlation between these two factors them- 
selves was .71. Intelligence correlated with the composite test of 
critical thinking at .48 and the reading scores correlated with critical 
thinking at .36. Although there was a tendency for the more intelligent 
groups to profit most from the training in critical thinking, there 
were individuals with intelligence scores of less than 100 found among 
those who profited most from the training in critical thinking. 

Maney (31), in a study of critical reading in science, found a 
correlation of .83 (significant at p .01) between scores of verbal 
intelligence and general reading comprehension; .75 between verbal 
intelligence scores and literal reading of science; and .67 between 
verbal intelligence and the critical reading of science. The influence 
of intelligence is shown clearly in the following data; general reading 
comprehension and literal reading of science correlate at .75, and 
general reading and critical reading at .60. With intelligence held 
constant, the first correlation becomes .35; the one involving critical 
reading scores falls to .11. The findings from Sochor's study (45) in 
social studies agree with Maney 's findings. 

Thus, it appears from the research studies reported here that 
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attitudes, Intelligence and general reading do affect ability to read 
critically. 



Teaching Critical Reading 



Only a few studies have been reported In which an attempt was made 
to teach critical reading per se to elementary or high school students. 
Among these are the ones by Kay (26) and Nardelll (34)* Using a pretest- 
posttest design, Kay measured gains of 385 high school students who 
were Instructed on four dimensions of critical reading. She found that 
between 14 and 22 per cent of the students did gain In their ability to 
form their own conclusions, to discern the author's purpose and to make 
comparisons of conflicting or correlating Ideas by one or several 
authors. Furthermore, less than 6 per cent of the students were able 
to improve in their ability to discover inaccuracies. Inconsistencies 
and omissions of essential information. Since a control group was 
lacking It Is impossible to tell If the results were due to the treat- 
ment or some other factor. 

Five experimental and three control groups consisting of sixth- 
grade classes were matched on reading ability, chronological age, I.Q,., 
and Initial creative reading ability in the research of Nardelll. Tests 
to measure abi 1 1 ty to (1) interpret authors' suggestions, (2) interpret 
feelings, and (3) recognize propaganda devices were administered. Lesson 
units to improve the abilities thus tested were designed by the re- 
searcher and taught by him to the experimental groups. A statistically 
significant mean gain for the experimental group was obtained, with 
the major gains In the area of recognizing propaganda devices. 

These two studies are representative of the research on the teaching 
of critical reading. Although the research Indicates that aspects of 
critical reading can be taught in the upper elementary and high school 
grades, the studies display certain inadequacies such as a lack of con- 
trol in the research design, or inadequate statistical treatment of 
the data. 

Closely related to research in teaching critical reading is a more 
comprehensive study directed toward teaching another skill, critical 
listening. Lundsteen (29) compared a group of fifth and sixth-graders 
(N ® 287) who had nine weeks of Instruction in critical listening with 
a group who followed the usual English curriculum. The lessons focused 
on (1) detecting the speaker's purpose, (2) analyzing and judging propa- 
ganda, and (3) analyzing and judging arguments. Critical listening was 
defined as the process of examining ideas, comparing them with some con- 
sensual data, and acting or concluding upon the judgment made. In the 
two-group experiment a significant difference (.01 level) was found in 
favor of the experimental group exposed to lessons in critical listening. 
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The study undertaken by the authors of this report builds on the 
findings of the studies reviewed In critical reading and critical 
thinking. However, It differs from most In that the materials were 
developed after extensive observations; experimental teaching extends 
down to grade one; the lessons Incorporate varied and unique aspects 
of critical reading; and the experimental teaching was done for an 
entire academic year. 



CHAPTER III 



PROCEDURES OF THE STUDY 



Developmental Phase 



Several necessary developmental activities were undertaken prior 
to the experimental phase in the Critical Reading Project.^ These 
activities included refinement and verification of the definition of 
critical reading, observations of classrooms to pilot test techniques 
and materials, development of an observation instrument, development of 
teaching units for the experimental treatment, and development of a test 
of critical reading. 



Refinement and Verification of the 
Definition of Critical Reading 

Preliminary to the development of materials for the experimental 
treatment and test items for the measuring instruments, it was necessary 
to develop an extensive operational definition of critical reading. In 
the initial planning stage, the investigators identified specific skills 
that they believed a mature critical reader would exhibit. This list of 
critical reading skills (see Appendix B) was stated in behavioral terms 
and was based on a definition proposed by Helen Robinson (38) and a three- 
factor construct described by David Russell (4l). During the initial 
phase of the project, this list of skills was sent for validation to 
fourteen reading experts across the country.* The panel of experts 
was asked to critique the list, to rate the importance of each skill, 
and to suggest any other skills they believed contributed to critical 
reading ability. Following their recommendations the list of critical 
reading skills was revised. During the pilot observation study (described 
in Appendix A) this list was checked for completeness in elementary 
school classrooms. When critical reading behaviors were observed other 
than those already included in the definition, they were added to the list. 



*The panel of reading experts to whom the tentative definition 
of critical reading was sent included: David Russell, Nila Banton Smith, 

Russell Stauffer, Donald Cleland, Helen Robinson, Constance McCullough, 
Sterl Artley, Robert Ennis, William Sheldon, Albert Harris, John DeBoer, 
William Eller, Ruth Strickland, and Roy Kress. 
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The following skills for the critical reading of all types of materials 
Including Informational, persuasive, and literary were Identified: 

I. Analysis and Evaluation of Informational and Persuasive Material 



A. Semantics In Writing 

1. Distinguishing between vague and precise words. 

2. Recognizing the difference between connotative and 
denotative meanings of words. 

3. Recognizing the persuasive use of words through such 

devices as: name cal 1 Ing,jl I tter ing generalities, 

and plain folks. 

4. Evaluating the effectiveness of the use of words according 
to the author's purpose. 

B. Logic In Writing 

1. Recognizing and evaluating the validity of writing. 

a) Examining the validity of an argument, I.e., 
judging whether conclusions necessarily follow 
from premises. 

b) Classifying Into groups and sub-groups. 

c) Determining appropriate use of all, some, and 
none statements. 

d) Discovering unstated premises and conclusions. 

2. Recognizing and evaluating the reliability of printed 
materials . 



a) 

b) 

c) 



d) 



e) 

f) 

g) 



Discovering ways to test the reliability of information. 
Determining soundness of premises and conclusions. 
Detecting material fallacies, e.g., hasty generali- 
zations, unrepresentative generalizations, faulty 
causal generalizations, post hoc reasoning, false 
analogies, false dilemmas, fallacies of composition 
and division, and all or nothing statements. 

Recognizing illogical reasoning in persuasive wri ting, 
e.g., testimonial, identification and transfer, band 
wagon, card stacking. 

Recognizing and evaluating different forms of infor- 
mational and persuasive writing. 

Distinguishing between objective and subjective evidence. 
Judging the reliability of information. 




C. Authenticity of Writing 



1. Recognizing adequacy of Information or the necessity of 
suspending judgment. 

2. Comparing relevant Information from multiple sources to 
recognize agreement or contradiction. 

3. Recognizing authoritative sources and evaluating them 
according to established criteria. 

4. Evaluating the qualifications of the author. 

5. Recognizing the publisher and sponsor's commitments. 

II. Analysis and Evaluation of Literary Material 
A. Literary Forms 



1. Recognizing characteristics of various genre of fiction, 

such as: fantasy, realistic fiction, historical fiction, 

and biography. 

2. Distinguishing among variants of a particular form of 

fiction. For example, distinguishing between various forms 
of fantasy: make-believe, fairy tale, folk tale, modern 

fantasy, fable, myth, science fiction, allegory. 

3. Developing criteria for evaluating each type of fiction. 

4. Recognizing the characteristic forms of poetry, e.g., 
narrative, lyric, haiku. 

5. Developing criteria for evaluating poetry. 

B. Components of Literature 



1. Identifying and evaluating characterization. 

a) Distinguishing between character delineation and 
character development. 

b) Recognizing ways the author reveals character. 

c) Developing criteria for assessing characterization. 

d) Comparing and evaluating methods of character develop- 
ment in two books. 

2. Identifying and evaluating plot structure. 

a) Recognizing the structure of the plot: accumulative, 

episodic, parallel . 

b) Tracing the development of plot structure: the sequence, 

the climax, denouement. 

c) Recognizing ways of attaining the climax: suspense, 

surprise, size and color of pictures. 

d) Recognizing and evaluating effectiveness of special 

techniques of plot development: foreshadowing and 

flashback. 
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3. Identifying and evaluating setting. 

a) Recognizing the elements of setting; place, time. 

b) Understanding the relationship of setting to action 
and character development. 

4. Identifying and evaluating theme. 

a) Distinguishing between theme and plot in a story. 

b) Identifying the story theme and comparing themes in 
several books. 

c) Evaluating effectiveness of theme presentation. 

C. Literary Devices 

1. Identifying and evaluating author's use of language. 

a) Interpreting and evaluating figurative language; 
metaphor, simile, personification. 

b) Evaluating the use of dialogue and authentic speech. 

c) Evaluating the author's style of writing. 

d) Interpreting symbolism and judging its effectiveness. 

2. Identifying and evaluating mood of writing. 

a) Recognizing the mood of selected poems and stories. 

b) Recognizing different ways the author achieves humor; 
surprise, slap-stick, exaggerations, anachronism. 

c) Recognizing the effective use of satire or irony. 

3. Identifying and evaluating point of view. 

a) Recognizing the point of view from which the story 
is told. 

b) Considering how the story would be different if told 
from another point of view. 

c) Comparing books written from different points of view. 



Pilot Observation Study 

The major purposes of the pilot observation study were to obtain 
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Ideas for materials and techniques for teaching critical reading and 
to check the completeness of the definition of critical reading be- 
haviors. in order to carry out these purposes, thirty teachers were 
observed who had been Identified as teaching critical reading. 

An observation instrument was developed with three categories 
for teacher verbal behavior: gathering information , refining -clarifying, 

and applving-evaluating . When the teacher asked a question requesting 
a specific answer from the pupil which was available in the reading 
material or class discussions this behavior was labeled gathering 
information. If a question required the pupil to use or evaluate infor- 
mation from the reading material it was labeled applying-evaluating . 

Such questions included those asking pupils to hypothesize, infer and 
evaluate. Questions which were categorized as refining-clarifying 
asked for refinement of previously-discussed ideas or Information which 
had been misinterpreted by the students. 

Pupils' responses were classified as critical or non-cr i t i ca 1 , 
depending upon the type of thinking exhibited. A response was defined 
as non-cr i tical If It could be drawn directly from the material being 
used. These Included factual answers, literal comprehension, verbatim 
reporting and repeating responses previously made by the teacher or 
another student. A response was recorded as critical if the student 
went beyond the literal meaning, i.e.. If he inferred, interpreted, 
extrapolated from the facts, detected logical fallacies in the material, 
or evaluated. 



Two observations were made of each teacher. A lesson prepared 
by the teacher was taught during the first observation and a lesson 
prepared by the critical reading staff was taught during the second 
observation. Analysis of records of pupil responses revealed that the 
teachers did elicit critical responses from their pupils during the 
first lesson, but that the number of critical responses elicited was 
increased by the specially-designed lesson plans. The data were 
analyzed to determine the number of critical responses elicited by 
each type of question. The analysis showed that applying-evaluating 
questions viere more effective for producing critical responses than 
questions used to gather information or refine and clarify information. 
Information-gathering questions were least effective for producing 
critical responses but seemed to be necessary in lessons directed 
toward critical reading. 

Although the pilot observation study was not rigorously controlled 
and thus the results lack internal and external validity, certain trends 
and indications were noted which provided ideas for the subsequent 
phases of the critical reading study. For example, in order to obtain 
critical responses from pupils, it appeared necessary for teachers 
to establish a background of information early in the lesson. When 
teachers moved directly to apply ing-eva 1 uative questions without 
establishing the substantive knowledge on a topic, they obtained 
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non-critical responses from a majority of the students. However, when 
teachers established a background of information and refined or clari- 
fied before asking applying-eyaluative questions, critical responses 
were more likely to occur. A complete presentation of this study can 
be found in Appendix A. 

It was impossible to classify all of the verbal behavior related 
to critical reading on the observation scale developed for the pilot 
study. Therefore, a new observation instrument was devised with ex- 
panded and more precise categories on both the teacher and pupil di- 
mensions. This scale was revised through several preliminary obser- 
vations of classrooms and the use of tape recorded discussions of 
reading material. A description of the observation instrument used 
in the experimental phase of the critical reading study is found in the 
fol lowing section. 



Development of the Observation 
Instrument Used in the Study 

The observation instrument used in the experimental study was 
also composed of two related category systems. Teachers' verbalizations 
were classified in one system, which was arranged vertically along the 
left side of the scale, and pupils' responses were coded in the other 
system, which was placed horizontally across the top of the scale (see 
Appendix E) . 

Bloom's (7) approach to ways of ordering knowledge was considered 
in developing the classification system for the teachers' verbal be- 
havior. Inasmuch as teachers assume the primary role of structuring 
discourse in the classroom and are usually concerned with both content 
and process objectives when teaching reading, it was reasoned that the 
Bloom categories would be useful in coding the teachers' structure 
of the reading-discussion lessons. Not all of the categories identi- 
fied by Bloom were used, and others were combined or re-named; however, 
his work was influential in defining each category. The eight teacher 
categories which consisted of gathering specific facts, clar ifyi ng, 
interpreting, analyzing, applying, summarizing, evaluating and controlling 

are defined in Appendix E. 

The main criterion in determining the pupil categories was the 
differentiation of levels of thinking that were evident in the pupils 
responses. Here the mental operations identified by Guilford (23) in 
his structure of the intellect proved useful in defining the separate 
types of thinking related to critical reading. Guilford (23) describes 
five major groups of intellectual abilities; memory, cognition, con- 
vergent and divergent thinking, and evaluation. These were adapted 
for this study and arranged in a continuum with random responses at 
Level 1, memory and cognition grouped into Level 2, convergent thinking 
designated as Level 3, divergent thinking as Level 4, and evaluative- 
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thinking as Level 5* Responses were recorded at the lower end of the 
continuum (Level l) when they evidenced guessing or random thoughts. 
Responses that showed literal cognition, memory, or repeating information 
directly from the reading source or earlier discussions, were placed 
at the literal level (Level 2). When children made inferences, re- 
organized reading material or extended the material through appropriate 
illustrations, these responses were recorded at Level 3. Responses 
were placed in Level 4 when children generalized, theorized, or hypothe- 
sized, or made unique application of the material read. Level 5 was 
reserved for responses that showed pupils had made an evaluative 
judgment, based upon established criteria that were stated. Responses 
at Levels 4 and 5 were considered to be most closely related to the 
act of critical reading. 



Development of Teaching Units 

Twelve units were developed for the experimental phase of the 
study: six for the experimental groups and six for the control groups. 

For each unit, background information was provided for the teacher. 
This information included an explanation of the unit content including 
its purposes, any unique features of the lesson plans, an explanation 
of difficult concepts, and any general teaching techniques running 
throughout all lesson plans in the unit. The lesson plans included a 
purpose statement, a list of the materials to be used, procedures to 
be followed and specific questions to be asked. Examples of these 
lessons are given in Appendix C. 

The experimental teaching units were based on information obtained 
during the pilot observational study, on information in the literature, 
and on past experiences of members of the critical reading staff. 

Basic considerations in the teaching units were as follows: (l) the 

student should have a background for the concept being developed or the 
background should be provided in the lessons, (2) the lessons were 
gradated in difficulty and the concepts presented built on previous 
lessons, (3) questions purposely allowed for diversity of opinions, 
and (4) students were expected to substantiate their comments or answers 
with evidence. 

Six teaching units were developed for the experimental groups: 
three suggested ways of working with informational and persuasive 
materials and three suggested ways of working with literary selections. 
Critical reading skills identified in the definition were grouped into 
the following teaching units: semantics, logic, general authenticity 

skills, literary form, components of literature, and literary devices. 

The teaching units for the control groups followed a similar 
format but were not intended to develop evaluative reading skills. 
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The units for these groups used a wide variety of children's books to 
enrich the various areas of the curriculum including social studies, 
mathematics, creative arts, science and literature. For example, in 
the mathematics unit, children's books were used to help develop the 
concepts of number, shape, size, time, and measurement. 

All of the teachers participated in the development of at least 
one of the lesson plans actually used. These teacher-prepared lessons 
were duplicated and distributed with the staff-prepared lessons. Because 
of the voluminous amount of lessons and materials to be developed for 
the experiment, it was necessary for the development to be continuous 
throughout most of the experimental phase. 



Development of The Ohio State University 
Critical Reading Test 

Since there was limited instrumentation in critical reading at 
the elementary school level, development of critical reading tests was 
necessary. Items were written to test many of the critical reading 
skills cited in the definition. 

Due to the long list of critical reading skills and the practical 
problem of staying within reasonable time limits for elementary school 
children, not all of the skills listed in the definition could be 
tested. When two skills were similar, items were written for only one 
and only those skills seem i ngly appropr i ate to each grade level were 
included. Parallel items were written for each of the skills selected 
and were tested with readability formulas. Items were refined and 
selected for the trial forms on the basis of test specifications and 
criteria developed by The Test Development Center at The Ohio State 
University. The trial forms for the primary and intermediate grades 
were then administered in the spring to a national sample of 3017^ 
children. Item analysis was performed on the results of this adminis- 
tration in order to select items for the final forms of the. test. 
Criteria for the selection of items included level of item difficulty, 
balance among incorrect alternatives, and discriminating power. Several 
changes in vocabulary were made as a result of empirical evidence from 
the trial forms. The reliability of each form at each grade level was 
above .80 and was regarded as acceptable. 

Development of Final Forms of the Critical Reading Test . 

Three forms of The Ohio State University Critical Reading Test (55) were 
developed from the items on the trial forms. The Level 2 Primary 
Critical Reading Test was intended to be used with children in grades 
two and three. The Level 1 Primary lest is basically the same test 
with a lower readability level and was developed to be used with the 
first grade children. The Intermediate form was intended for use in 



grades four, five, and six. A 4.0 reading level is necessary to master 
the general reading mechanics of the Intermediate form. Sample items 
from the Primary and Intermediate forms are provided In Appendix D. 

Norminq Final Forms of the Critical R eading Test. The Test 
Development Center selected another national sample for the purpose of 
norminq the tests. Forty-six school systems from four major geographical 
areas contributed to the normative sample. A random sampling was made 
of schools listed in the state educational directories. The states 
which contributed to the normative sample, with the number of schools ^ 
and classes in parentheses after each state, were Idaho, (7: IS) ; Louisiana 
(4:10): Maine, (7:14); Minnesota, (6:16); Mississippi, (5:|0); New 
Jersey, (6:16); Oklahoma, (4:9); and South Carolina, (7:20). The total 
number of students included in the fall norms is 3123 and in the spring 
norms is 1868. The means and standard deviation at each grade level 
obtc ined for the national sample are presented in Table 1. 



TABLE 1 

THE OHIO STATE UNIVERSITY CRITICAL READING TEST 
MEANS AND STANDARD DEVIATIONS 
NATIONAL SAMPLE 





Fall 


Spr i ng 


Test 


Grade 


N 


Mean 


Standard 

Deviation 


N 


Mean 


Standard 

Deviation 


Level 1 Primary 


1 


570 


13.79 


5.15 


334 


18.49 


6.90 


Level 2 Primary 


2 


485 


13.88 


5.52 


321 


17.72 


6.79 


Level 2 Primary 


3 


513 


20.07 


7.85 


341 


22.67 


8.40 


1 n termed i ate 


4 


516 


17.22 


5.54 


301 


21.55 


7.92 


1 ntermediate 


5 


522 


21 .23 


6.80 


276 


24.78 


8.20 


1 ntermediate 


6 


517 


25.00 


7.69 


295 


27.21 


8.42 

f 



Reliability of the Final Forms . The Kuder-Ri chardson formulae 
20 and 21 and the split-half coefficients were used to check the re- 
liability of the tests at each grade level. These rel iabi 1 i t ies as 
well as the standard error are given in Table 2 for the fall and spring 
administrations of the test. 
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Validity of the Final Forms . An attempt was made to determine the 
concurrent validity of the test six months after the experiment by asking 
teachers selected randomly from the control and experimental groups to 
list the five highest and five lowest critical readers in their group 
according to their judgment. Their ratings were compared to the ratings 
made from the scores on The Ohio State University Crit ical Reading Test. 
When the teacher judgments were compared with the results of the test, 
the teachers selected three of the five highest and two of the five 
lowest critical readers according to the test. The probability that 
this would occur by chance is .16 to .02 (hyper-geometric probability 
distribution). It is probable that the teachers' judgments would have 
more closely paralleled test results if the ratings had been made 
immediately following the experiment. 

In the body of this report, data are presented which provide 
evidence of construct validity. Since the collection of data for con- 
struct validity was necessarily collected over a long period of time, it 
was done concurrently with the experimental phase. Correlational data 
presented in Chapter IV provide the evidence used to establish construct 
validity. 

Factor Analysis of the Critical Reading Test . Factor analysis 
of the spring norms of The Ohio State University Criti cal Reading Test 
was done for the following purposes: 

1. To clarify what the test measures. 

2. To obtain factorial validity of the test. 

3. To better understand the nature of critical reading. 

4. To find specific areas in which the test needs improvement. 

Hotelling's Principal Axis Method of factoring the correlation 
matrix was used in the first factor analysis. In view of the non- 
availability of outside criteria, it was decided to factor analyze the 
42 X 42 matrix of inter- item correlations. This was accomplished 
through the use of the "OSD Factor Analysis Program (FACANA)" and The 
Ohio State University Computer Center's IBM 70Sl4 computer. After 
twenty iterations, the entries in the residual matrix appeared to be 
small. The twenty factors thus obtained were then rotated according to 
Kaiser's Varimax Method. The first-grade data, which is illustrative 
of other grades, showed that (a) the total fractional contribution of the 
twenty factors is equal to .7090, and (b) the contribution of the first 
rotated factor is .0605 and that of each of the remaining nineteen factors 
varies between .03 and .04. 

When the results obtained by the Principal Axis Method were 
examined, it was decided to study the data by using the Hierarchical 
Method in order to identify possible group factors. Since the factor 
analysis was not a part of this proposed study and is not yet finished, 
the results will be published elsewhere. 
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Experimental Phase 



General Procedure 

In the summer of 1965> before the experimental phase, two training 
workshops were conducted— one in critical reading and one in children's 
literature. Teachers for the study were volunteers from these workshops. 

In September, pretests in general reading and critical reading were 
administered to subjects in grades two through six. Each teacher was 
then given an experimental teaching unit (either critical reading or 
children's literature) and instructed to teach two lessons per week. 

The remaining units were periodically sent to the teachers throughout 
the year. Teachers were observed systematically during the teaching 
of the units and assistance was given to the teachers in both the experi- 
mental and control groups through individual conferences and two one- 
day training sessions in the fall and in the middle of the year. At 
the end of the academic year, the critical reading and general reading 
tests were administered again. Indications of intelligence and personality 
were obtained through tests administered in January (l*Q.«) ^nd May 
(Personality). The procedures for the subjects in grade one were the 
same except pretests of the criterion measures were administered in 
January. Since first-grade subjects in a pilot sample had difficulty 
reading the critical reading test, the test was read aloud to the first 
graders in the study while they followed along marking their answers. 

This procedure was repeated in May. 



Sample 



Subjects for the study consisted of 651*children in grades one 
through six from seven public school systems In Franklin County, Ohio. 
Twenty-four intact classroom groups were involved, four at each grade 
level. Two of the classes were assigned to the control group and two 
were assigned to the experimental group at each grade level. Since the 
study was to be conducted over an entire academic year, the teachers 
could not be selected randomly, but instead were chosen from teachers 
who attended the summer training workshops in \$ 65 . The experimental 
teachers were chosen from the Critical Reading Workshop and the control 
teachers were chosen from the Children's Literature Workshop. Super- ^ 
visors from The Ohio State University worked cooperatively with admini- 
strators in the Franklin County School Systems to identify potential 
teachers and to encourage them to attend the workshop and participate 
in the experiment. 

Thirty-eight teachers volunteered from the Children's Literature 
Workshop and twenty-four volunteered from the Critical Reading Workshop. 
In selecting from among them, an attempt was made to choose teachers 
with classes from comparable socio-economic levels. For example, if 
in grade three one experimental group was from a high socio-economic 

*An additional fifty-one students moved during the year and were 
dropped from the sample. This loss of 7 P®i" cent Is considerably lower 
than the normal 15 P©r cent in the entire Columbus Public School System. 

V 
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area while another was from a low socio-economic area, the control . 
groups were chosen accordingly. Tests of homogeneity of regression 
were conducted between the total treatment groups (two groups combined ^ 
in each treatment) at each grade level on all of the criterion measures. 
Except in two’’^ cases out of thirty, the regressions were homogeneous, 
thus indicating that there was no sampling bias. 



Design of the Study 

The design for the analysis of the effects of the program of critical 
reading instruction at each grade level was a two factorial 2X2 (repli- 
cations by treatment) pretest-posttest design. The technique utilized 
for analyzing the data was the analysis of covariance. The design was 
as follows: 



Rl 



R2 



Ti T2 Ti T2 

XYXY XYXY 

where: R * Repl i cation 

T Treatment 

T| ® Experimental 
T2 ® Control 
X » Covariate: pretest 
Y s Variate: posttest 

As mentioned previously, four intact classroom groups were studied at 
each grade level. Two of these groups were assigned to the control 
group and two were assigned to the experimental group. The subjects 
in the experimental group received instruction in critical reading while 
the subjects in the control group received instruction in children's 
literature in order to minimize the Hawthorne effect. 

To assist in determining whether factors other than the treatment 
affected the results of the study, the investigators conceptualized the 
groups at each grade level as replication groups. There were two repli- 
cations of this study at each grade level. Thus the research was conducted 
with one experimental and one control group (Replication one) and then 
repeated with another experimental and control group (Replication two). 



*A11 variables and tests are explained in the I nstrumentati'on section. 

’’^^General reading in the second grade and the Literature sub- test 
of critical reading in the third grade. 
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within treatments, the groups at each grade level were randomly assigned 
to replications. The analysis of replication differences provided a 
statistical measure of the experimental effect when the experiment was 
independently repeated. 

In order to further control for initial differences, the technique 
of analysis of covariance was employed. Pretest measures were made 
on each subject to determine initial ability in critical reading and 
general reading. This score was used as a covariate of the posttest 
score. In this way, the effect of initial ability on the results was 
statistically eliminated. 



Teaching Plan 

The teaching plan for the experimental groups on most of the 
lessons included the following: (1) reading the materials, (2) discussing 

the factual content, (3) establishing the criteria for evaluating through 
discussion, illustrations, or questions, (4) asking students to use 
the criteria to evaluate the printed materials, and (5) assigning follow- 
up activities such as writing, comparing books, and searching for 
evidence. This sequence had been established through an earlier pilot 
observation study (see Appendix A) as one conducive to eliciting critical 
responses from children. 

The teaching plan in logic differed somewhat from the rest since 
reading materials lending themselves to such analysis were scarce. 

Hence, these lessons were limited to the following activities: (l) 

establishing criteria for using logic in evaluation, (2) applying the 
criteria to worksheets and any available materials containing the 
various fallacies, (3) checking pupils' accuracy through questions and 
discussion, and (4) assigning follow-up activities again applying the 
criteria to the reading of other materials whenever it was possible. 

The teaching plans for the control groups included the following: 

(1) reading the materials, (2) discussing the factual content, (3) 
asking children to relate the story to the content area of the units, 
e.g., mathematics, and (4) assigning follow-up activities such as writing 
and additional reading. Thus, the format of the teaching plans for all 
groups was the same except that in the experimental group criteria for 
evaluating were established, and the questions and activities required 
the children to use the criteria to evaluate printed materials. 



I nstrumentatlon 



The tests other than The Ohio State University Critical Reading 
Test administered to all subjects were as follows: 
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(1) California Achievement Tests !n Reading (50, 

Form W in the fall and Form X in the spring. 

Scores were obtained on Total and on Vocabulary and 
and Comprehension; 

(2) The Lorqe-Thorndike Intelligence Tests (28), 

Levels 1 and 2, Form A for grades one and two 
and Levels A-D, Form 1 for grades three through 
six. This test provides a measure on non- 
verbal ability for all six grade levels and 

a measure of verbal ability for all grades 
except one and two; 

( 3 ) The California Test of Personality (50), 

Primary Form AA for grades one, two, and three 
and Elementary Form AA for grades four, five, 

and six. Scores used on this test were Personality 
Total and the subscores of Personal and Social 
Adjustment.* 

The reliabilities of the standardized tests and the method by 
which they were obtained are given in Table 3* All instruments were 
administered in groups. 

The criterion instruments for critical reading were the tests 
developed by the project staff. Four scores were obtained on the 
critical reading test— a total score and scores from items grouped into 
three sections: Logic, General and Literature. The items that tested 
the subject's ability to detect fallacies and propaganda techniques 
used in printed materials and his ability to evaluate the internal 
consistency of an argument were labeled as Logic. I terns that evaluated 
the subject's ability to identify the author's and publisher's point 
of view and biases, to judge the author's qualifications and to make 
comparisons of related content from various sources were categorized 
as General. I terns measuring the subject's ability to identify literary 
forms and to analyze and evaluate story structure, character develop- 
ment, story setting, format and theme of the story and the author's 
use of literary devices were classified under Literature. The total 
number of items for the critical reading test and its sections were 
as f ol lews: 





Logic 


Genera 1 


Li terature 


Tota 1 


Level 1 Primary 


17 


10 


15 


42 


Level 2 Primary 


17 


12 


17 


46 


1 ntermed iate 


21 


15 


18 


54 



*At the request of the test committee of the U. S. Office of 
Education, several items on family relationships were deleted from 
the Social Adjustment section of the test. 
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RELIABILITY OF STANDARDIZED TESTS USED IN THE STUDY 
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Observations 



In order to determine what types of teacher questions produced 
critical responses from subjects, six observations were made at regular 
intervals in each of the twenty-four classrooms resulting In a total of 
144 observation records. Three observers were trained to use the 
observation scale (described on pages 24 and 25) through repeated visits 
to classrooms and extensive use of tape recordings. On-the-spot categori- 
zation was made of both the teacher's verbal expressions and the pupils' 
responses. Inter-observer reliability was checked periodically by the 
Spearman- Brown Prophecy Formula. Coefficients for the teacher categories 
ranged between .67 and .97 with a mean of .84 while inter-observer 
reliability for pupil categories ranged between .61 and .87 with a mean 
of .73. Two observers participated in each observation. While one 
observer classified the verbal behavior of both teachers and pupils, the 
other kept a companion record that identified by number and sex each 
pupil who spoke. This coding provided data about the number of different 
pupils who participated in the discussions and the degree of partici- 
pation of each of the sexes. The teachers were informed of the obser- 
vations in advance and the time of each observation was limited to 
twenty-five minutes. 



CHAPTER IV 



PRESENTATION AND ANALYSIS OF DATA 



Introduction 



The presentation and analysis of the data froin this study are given 
In this chapter in three major sections: the feasibility of teaching 

critical reading, factors related to the ability to read critically and 
observations of the verbal interaction during critical reading lessons. 
The final section of the chapter presents the teachers' reactions to 
teaching critical reading. 



Feasibility of Teaching Critical Reading 



The major purpose of the study v;as to determine whether or not 
children in the elementary grades could be taught to read critically 
while normal progression In other basic reading skills was maintained. 

In order to answer this question, data including the scores from the 
criterion measures of critical reading and general reading were analyzed 
using the technique of analysis of covariance (ANCOVA) . Results are 
reported here for the total scores on The Ohio State University Critical 
Reading Test (hereafter called Critical Reading Total) and the Cal ifornia 
Achievement Test in Reading (hereafter called General Reading Total) as 
well as the sub-test scores of Logic, General, and Literature from the 
critical reading test. The complete ANCOVA tables for these data are 
presented in Appendix F. The data on treatment, replication and inter- 
action effects at each grade level are presented first with differences 
across grade levels reported immediately following. There are several 
instances where replication or interaction effects occur in the data. 

These effects indicate that there was some source of variation, other 
than the treatment, throughout many of the grades. An explanation of 
the interaction effects is given on page 46. Summaries are also presented 
for the data at each grade level on page 48 and for the data across 
grade levels on page 55- 
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Analysis of Scores on Criterion 
Mea s u res at Each Grade Level 



Grade One . The mean score of the experimental group on the Critical 
Reading Total was significantly higher (p < .01) than that of the control 
group. This is shown by the F tests from the analysis of covariance 
presented in Table 5* The unadjusted means and gains on the criterion 
measures for grade one are presented also in Table 4. No main effect 
of replications was observed^ however, there was a significant replication 
by treatment interaction (p <. .01) due mainly to the high scores of the 
control group in Replication one. 

On the sections of the critical reading test, the experimental 
group scored significantly higher (p .01) than the control group on 
Logic. They also scored higher on Literature but at a marginal level 
of significance (p < .10) while the control group scored higher (p .10) 
on the General section. The main effect due to replications on the Logic 
scores indicates that there was some source of variation, not due to the 
treatment, between the groups on this sub-test. The interaction effects 
occurring on the three sections of The Ohio State Universit y Critical 
Reading Test are due mainly to the high scores of the control group in 
Repl i cat ion one. 

The experimental group also scored higher (p < .10) than the 
control group on the General Reading Total, but there was both a signifi- 
cant replications effect (p . 0 \) and a significant replication by 
treatment interaction (p<.01) again due to the high mean score of the 
control group in Replication one. 

Thus in grade one, the children who were given instruction in 
critical reading did better on the critical reading test than the children 
who did not receive such instruction. However, the significant replication 
and interaction effects that were observed indicate there was some source 
of variation causing one control group to make consistently high mean 
scores on the tests. 

Grade Two . The experimental group also had a higher mean score 
than the control group on the Critical Reading Test Total in grade two; 
however, the difference was at a marginal level of significance (p ^ .10). 
Table 6 contains the unadj‘usted means and gains on the criterion measures 
and Table 7 contains the F tests from the analysis of covariance for 
these data. It can be observed in the F table that there was neither a 
main effect due to replications nor a significant interaction effect 
on the Critical Reading Test Total. 

The differences between the scores of the experimental and the 
control groups were not significant on either the General or Literature 
sections of the critical reading test at this grade level. Only on the 
Logic section of the test did the experimental group score significantly 
higher (p<C.01) than the control group. There was also a replications 
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TABLE 4 



UNADJUSTED MEANS AND GAINS ON GENERAL 
AND CRITICAL READING TESTS 
GRADE ONE 



Replication Treatment 

R] R2 T|(Exper.) T2 (Control) 



CRITICAL READING TOTAL 



Pretest 


13.24 


13.13 


12.98 


13.38 


Posttest 


18.15 


19.78 


20.14 


17.66 


Ga i n 


4.91 


6.65 


7.16 


4.28 


Logic Section 


Pretest 


5.85 


4.76 


4.70 


6.00 


Posttest 


7.04 


8.26 


8.64 


6.56 


Ga i n 


1.19 


3.50 


3.94 


.56 


General Section 


Pretest 


3.04 


3.13 


3.24 


2.92 


Posttest 


4.43 


3.87 


3.84 


4.50 


Ga i n 


1.39 


.74 


.60 


1.58 


Literature Section 


Pretest 


4.35 


5.22 


5.04 


4.46 


Posttest 


6.69 


7.65 


7.66 


6.60 


Ga i n 


2.34 


2.43 


2.62 


2.14 


GENERAL READING TOTAL 


Pretest 


49.76 


40.76 


52.10 


39.14 


Posttest 


65.04 


62.76 


71 .24 


56.74 


Ga i n 


15.28 


22.00 


19.14 


17.60 



TABLE 5 

F TESTS FROM THE ANALYSIS OF COVARIANCE 

GRADE ONE 







CRITICAL 


READING 




GENERAL 

READING 

Total 


Source 


Total 


Logic 


General 


Literature 


Repl i cat ion 


2.65 


6.91* 


1.97 


1.55 


8.74'W* 


Trea tment 


7.66vWc 


19 . 6 h'«v 


3.63a 


3.42a 


3.8oa 


Rep. by Treat. 


25.75— 


1 8 . 28 ''*^'** 


7 . 10 v«v 




29.86v«v 



^Significant at the .10 level 
*Sign if leant at the .05 level 
'«*Sign if leant at the .01 level 
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TABLE 6 



UNADoOSTED MEANS AND GAINS ON GENERAL 
AND CRITICAL READING TESTS 
GRADE TWO 



Replication Treatment 

R| R2 T|(Exper.) T2(Control) 



CRITICAL READING TOTAL 


Pretest 


16.36 


15.67 


16.75 


15.41 


Posttest 


22.90 


23,48 


24.74 


22.08 


Gain 


6.54 


7,81 


7.99 


6.67 


Loaic Section 


Pretest 


5.92 


5.68 


6.26 


4.54 


Posttest 


7.76 


8.45 


9.36 


7.30 


Ga i n 


1.84 


2.77 


3.10 


2.76 


General Section 


Pretest 


4.68 


4,90 


. 4.81 


4.79 


Posttest 


6.08 


6.30 


6.36 


6.08 


Ga i n 


1 .40 


1 .40 


1.55 


1 .29 


Literature Section 


Pretest 


5.76 


6.03 


5.68 


6.08 


Posttest 


9.06 


8.65 


9.02 


8.70 


Gain 


3.30 


2.62 


3.34 


2.62 


GENERAL READING TOTAL 


Pretest 


65.74 


69.85 


67.98 


67.03 


Posttest 


83.34 


81 .80 


81.77 


83.13 


Gain 


17.60 


11.95 


13.79 


16.10 



TABLE 7 

F TESTS FROM THE ANALYSIS OF COVARIANCE 

GRADE TWO 



Source 


Total 


CRITICAL 

Logic 


READING 

General 


Li terature 


GENERAL 

READING 

Total 


Repl ication. 


.92 


5.90* 


.21 


.70 


9.77.V.V 


Treatment 


3.65a 


1 1 .73— 


.45 


.54 


1.71 


Rep. by Treat. 


.35 


.10 


.08 


.86 


.13 



^Significant at the .10 level 
*Signif leant at the .05 level 
**Slgnif leant at the .01 level 
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effect (p<.. 01 ) but no interaction effect. 

On the General Reading Test Total, there was no significant difference 
between the experimental and control groups. Although there was no 
interaction effect, there was a significant main effect of replications 
(p < . 01 ). 



In summary, the children in the experimental group in grade two 
did significantly better than the children in the control group on the 
Logic sub- test of critical reading. They also had a higher mean score 
on the Critical Reading Total, but at a marginal level of significance. 
The significant replication effects on Logic and General Reading Total 
indicate that some factor other than the treatment caused differences 
between replications. 

Grade Three . The experimental group scored significantly higher 
(p 4 ;;^. 01) than the control group on the Critical Reading Total (see 
Tables 8 and 9). There was no difference between replications 
but a significant interaction effect (p<.01) occurred. This inter- 
action was due to the high scores of the control group in Replication 
two. 



‘This was the only grade level at which the experimental groups 
scored significantly higher than the control groups on each of the 
sections of the critical reading test. There was a significant diff- 
erence (p <1 . 05 ) between replications only on the General section, but 
Interaction effects (p <. .01) occurred on each section. In each case 
the interaction was due to the high scores of the control group in 
Replication two. 

There were no differences between treatments or between repli- 
cations on the General Reading Total but a significant interaction 
effect (p<.01)was observed. The significant interaction effect was 
again due to the high scores of the control group in Replication two. 

In grade three, the children who had received instruction in 
critical reading did significantly better on the critical reading test 
than the children who had not received such instruction. The significant 
interaction effects and the main effect of replications on the General 
Reading and Critical Reading Totals mean that there was some factor 
other than the treatment causing the control group in Replication two 
to receive consistently higher scores than the other groups. 

Grade Four . In grade four (see Tables 10 and 11 ) the experi- 
mental group also had a significantly higher (p <C -05) mean score on 
the Critical Reading Test Total than the control group. However, there 
was a significant effect due to replication and interaction. These 
effects were due to the low scores of the experimental group in Repli- 
cation two. 
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TABLE 8 



UNADJUSTED MEANS AND GAINS ON GENERAL 
AND CRITICAL READING TESTS 
GRADE THREE 



Replication Treatment 

R| R2 T](Exper.) T2 (Control) 



CRITICAL READING TOTAL 


Pretest 


21.21 


18.78 


21.37 


18.32 


Posttest 


26.75 


24.12 


28.80 


21.98 


Gain 


5.54 


5.34 


7.43 


3.66 


Logic Section 


Pretest 


7.47 


6.53 


7.35 


6.57 


Post test 


10.00 


9.29 


11.10 


7.99 


Gain 


2.53 


2.76 


3.75 


1.42 


General Section 


Prete-t 


5.83 


5.24 


5.96 


5.04 


Posttest 


7.21 


6.02 


7.21 


5.88 


Gain 


1.38 


.78 


1.25 


.84 


Literature Section 


Pretest 


7.87 


7.02 


8.06 


6.71 


Posttest 


9.51 


9.20 


10.49 


8.11 


Gain 


1.64 


2.18 


2.43 


1.40 


GENERAL READING TOTAL 


Pretest 


52.95 


65.74 


72.58 


55.34 


Post test 


64.00 


78.80 


80.71 


73.94 


Ga i n 


11.05 


13.06 


7.13 


18.60 



TABLE 9 

F TESTS FROM THE ANALYSIS OF COVARIANCE 

GRADE THREE 







CRITICAL 


READING 




GENERAL 

READING 

Total 


Source 


JTota 1 


Logic 


General 


Literature 


Repl icati on 


.34 


.88 


6 . 20 * 


.14 


.63 


Treatment 


22 . 15 '’f* 


46 . 38 ^Wf 


5.52* 


8.52** 


1.56 


Rep. by Treat. 


9 . 80 ^bv 


l 4 . 88 >’«v 


14 . 80 ** 


l 6 . 85 ^w^ 


15 . 57 ’^^ 



*Sign if leant at the .05 level 

**Signif leant at the .01 level 



TABLE 10 



UNADJUSTED MEANS AND GAINS ON GENERAL 
AND CRITICAL READING TESTS 
GRADE FOUR 



Replication Treatment 

R] R 9 T|(Exper.) T 2 (Control) 



CRITKAU READING TOTAL 


Pretest 


15.79 


15.36 


14.87 


16.50 


Post test 


18.52 


16.61 


17.88 


17.22 


Gain 


2.73 


1.25 


3.01 


.72 


Logic Section 


Pretest 


6.06 


5.43 


5.82 


5.92 


Post test 


7.92 


6.55 


7.64 


6.89 


Gain 


1.86 


1.07 


1.82 


.97 


General Section 


Pretest 


4.61 


4.46 


4.13 


4.96 


Posttest 


4.97 


4.50 


4.74 


4.76 


Gain 


.36 


.04 


.61 


-.20 


Literature Section 


Pretest 


5.15 


5.41 


4.92 


5.62 


PosttGJt 


5.63 


5.43 


5.50 


5.57 


Gain 


.48 


.02 


.58 


-.05 


GENERAL READING TOTAL 


Pretest 


54.13 


48.77 


44.73 


56.78 


Post test 


69.77 


59.00 


59.85 


69.64 


Gain 


15.64 


10.23 


15.12 


12.86 



TABLE 11 



F TESTS FROM THE ANALYSIS OF COVARIANCE 

GRADE FOUR 



Source 


Total 


CRITICAL 

Logic 


READING 

General 


Li terature 


GENERAL 

READING 

Total 


^epl i cat ion 


5 . 54 * 


9.65’W^ 


1 .50 


.46 


6 . 19 * 


Treatment 


4 . 1 6>v 


3.60® 


.56 


.12 


.01 


Rep. by Treat. 


lO.llVrtV 




3.60 


1 .58 


9.44Vf* 



®Sign if leant at the .10 level 
^Significant at the .05 level 
**Signif leant at the .01 level 
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Although there were no differences on the General or Literature 
sections of the critical reading test in grade four, the experimental 
group did score higher (marginal significance level p<,.10} on the 
Logic section than the control group. On the Logic section there 
was also a significant difference (p<l..01) between replications 
and a significant Interaction effect (p < .01). This was due to the 
low scores of the experimental group In Replication two. 

There was no difference between the mean scores of the experimental 
group and the control group on the General Reading Total ; however, 
there were significant effects due to replication (p -^.05) ^^d to 
interaction (p <.0l). Again, this was due to the low scores of the 
experimental group in Replication two. 

Accordingly, the children in the fourth grade experimental group 
did better than the control group on the Critical Reading Test Total 
and -the Logic sub-test of critical reading. At the same time, they 
did as well as the children In the control group on the general reading 
test, it still remains necessary to explain the Interaction effects, 
i.e., why the experimental group In Replication two received relatively 

low scores. 

Grade Five . The mean score of the experimental group on the 
Critical Reading Total was significantly higher (p -C .01) than that 
of the control group in grade four (see Tables 12 and 13 )i hut there 
was a significant effect (p<C*05) due to replications. 

On the sections of the critical reading test, the experimental 
group scored significantly higher (p < .02) than the control group only 
on Logic. Replication effects (p<.0l) occurred only on the General 
section while interaction effects (p < *05) were observed in both the 
Logic and General sections. In '■.he Logic section, the interaction was 
due mainly to the high scores of the control group in Replication one 
while in the General section the interaction was due to the low scores 
of the experimental group in Replication one. 

There were no effects due to treatment on the General Reading 
Test Total, but a significant replications effect (p < .05) occurred. 

Thus in grade five, the children who had received instruction 
In critical reading did better on the Critical Reading Test Total and 
the Logic section than the children who had not received such instruction. 
However, the significant repl ication and interaction effects indicate 
that there was some source of variation confounding the results. 

Grade Six . Although the experimental group in grade six scored 
higher than the control group on the Critical Reading Test Total, it 
was at a marginal level of significance (p^.lO). The unadjusted 
means and gains on the criterion measures for this grade are reported 
in Table 14 and the F tests from the analysis of covariance for these 
data are presented in Table 15 • As may be observed in these tables. 
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TABLE 12 



UNADJUSTED MEANS AND GAINS ON GENERAL 
AND CRITICAL READING TESTS 
GRADE FIVE 



Replication Treatment 

R^ R2 T^(Exper.) T2(Control) 



CRITICAL READING 


TOTAL 










Pretest 




21 .96 


21.44 


20.33 


23.34 


Posttest 




25.62 


27.80 


26.75 


26.79 


Gain 




3 *66 


6.36 


6.42 


3.45 


Logic Section 












Pretest 




8.00 


8.10 


7.70 


8.32 


Post test 




10.13 


10.85 


11.69 


9.25 


Gain 




2.13 


2.75 


3.99 


.93 


General Section 










Pretest 




7.49 


6.89 


6.73 


7.43 


Posttest 




7.45 


8.38 


7.33 


8.59 


Ga i n 




-.04 


1.49 


.60 


1.16 


Literature Section 










Pretest 




6.47 


6.95 


5.90 


7.59 


Posttest 




8.04 


8.57 


7.73 


8.95 


Gain 




1.57 


1.62 


1.83 


1.36 


GENERAL READING TOTAL 










Pretest 




89.45 


76.79 


77.62 


88.34 


Posttest 




93.82 


89.38 


88 .28 


94.91 


Ga i n 




4.37 


12.59 


10.66 


6.57 






TABLE 13 








F TESTS FROM 


THE ANALYSIS 


OF COVARIANCE 








GRADE FIVE 












CRITICAL 


READING 




GENERAL 
READI NG 


Source 


Total 


Logic 


General 


Li terature 


r\L»ni^ 1 iiu 

Total 


Repl i cat ion 


5 . 46 * 


.55 


9 . 69 *v* 


.21 


6 . 85* 


Treatment 


7.1 hbV 


26 . 93 'W‘ 


1 .82 


.04 


.67 


Rep. by Treat. 


. 94 


5.43* 


3.99* 


.13 


.44 



*Sign if leant at the . 05 i level 
**Sign if leant at the .01 level 
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TABLE 14 



UNADJUSTED MEANS AND GAINS ON GENERAL 
AND CRITICAL READING TESTS 
GRADE SIX 



Repl Icatlon Treatment 

R, R 2 T, (Exper.) T 2 (Control) 



CRITICAL READING TOTAL .. .. 


Pretest 


25.83 


20.31 


22.55 


ZH.ZJ 


Posttest 


29.83 


26.58 


28.67 


27.91 


Ga i n 


4.00 


6.27 


6.12 


3.42 


Loqic Section 


Pretest 


10.09 


7. 73 


8.70 


9.3^ 


Post test 


11.47 


11.16 


11.99 


10.52 


Ga i n 


1 .38 


3.43 


3.29 


1 .20 


General Section _ 


Pretest 


8.13 


6.24 


7.10 


7.W 


Post test 


9.09 


7.64 


8.49 


8.38 


Gain 


.96 


1.40 


1.39 


.94 


Literature Section 


Pretest 


7.79 


6.31 


6.75 


7.47 


Post test 


9.25 


7.64 


8.19 


9.01 


Gain 


1.46 


1.33 


1 .44 


1 .54 


GENERAL READING TOTAL 


Pretest 


94.42 


76.38 


86.18 


86.19 


Post test 


100.51 


88.89 


95.35 


94.98 


Gain 


6.09 


12.51 


9.17 


8.79 



TABLE 15 



F TESTS FROM THE ANALYSIS OF COVARIANCE 

GRADE SIX 



Source 


Total 


CRITICAL 

Logic 


READING 

General 


Li terature 


GENERAL 

READING 

Total 


Repl i cat ion 


2.33 


2.33 


.00 


1.51 


.50 


Treatment 


3.75^ 


6 . 53 * 


.52 


.75 


.03 


Rep. by Treat. 


.26 


1 .03 


.46 


.18 


• 81 



^Significant at the .10 level 
*Sign if leant at the .05 level 
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there were no differences on either the General or Literature sections 
of the critical reading test; but the experimental group did score 
significantly higher (p<..05) than the control group on the Logic 
section. As was true in most of the other grades, no differences were 
found on the General Reading Total scores for grade six. 

Explanation of the Replication bv Treatment Interactions . As 
mentioned In the Procedures Chapter, intact classroom groups were used 
in the sample. Since the study was conducted over an entire academic 
year, i t was Impossible to choose classrooms randomly, but instead 
teachers were chosen from among volunteers. This made it necessary 
to control for initial differences in the groups; therefore, the technique 
of analysis of covariance was employed. Prior to using this technique,' 
a check was made on the comparability of the regression lines of the 
combined treatment groups. This test showed that the regressions were 
homogeneous thus indicating that there was no sampling bias and that 
all main effects of treatment observed are due to the experimental 
manipulations and not to other sources of variation. As a further 
check on sampling differences, the covariance design employed divided 
the groups into two replications. Each of the experimental and control 
groups was randomly assigned to a replication. When the data were 
analyzed, a number of replication by treatment (R x T) interactions 
occurred. These interactions could have been due to intelligence, 
teachers' ability or some other uncontrolled variable. 

Because intelligence scores for each group were available, Newuian 
Keuls multiple range tests were applied to the I .Q... means at each grade 
level to determine whether this factor might have contributed to the 
R X T Interactions. It appears from an inspection of Table 16 that 
many of the interactions are due to intelligence differences. An 
explanation of these interactions at each grade level is given below. 

In grade one where R x T interactions occurred on every criterion 
measure, the 1 .0,. mean of the control group in Replication one was 
significantly higher than that of the experimental group in Replication 
one. An inspection of the interactions shows that it was the high 
scores on the criterion measures in that group that made the mair. 
contribution to the interaction. 

There were no I .Q, differences in grade two and there were no 
R X T interactions. In grade three, where there were R x T interactions 
on every criterion test, the interactions were due to the low scores 
made by the control group in Replication one and the fairly high scores 
of the control group in Replication two on the criterion measures. 

Multiple range tests indicated that the mean 1 .0.. score of the former 
group was significantly lower than the mean I .CL. scores of all the other 
groups at that grade level while the mean 1 .0,. score of the other 
control group was higher than one experimental group but equal to that 
of the other experimental group. A comparison of the mean scores on 
the criterion measures between this control group and the two experimental 
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groups shows that the experimental group with equal I .Q. scores had a 
higher mean while the experimental group with lower I .Q,. scores had a 
lower mean. 

Interactions occurred on the criterion measures of General Reading 
Total, Critical Reading Total and the Logic section of the critical 
reading test in grade four. These interactions were due to the extremely 
low mean scores of one experimental group. When the mean intelligence 
score for this group was examined, it was determined that the group 
had a significantly lower I .d. score than did any of the other three 
groups. There were no differences in the I .Q,« scores among the other 
three groups. 

In grade five, interactions occurred on only two criterion measures— 
the Logic and General sections of the critical reading test. Since 
there were no I .Q,. differences among the groups at that grade level, 
the differences may be attributable to teacher ability or some other 
uncontrolled factor. 

No interactions occurred in grade six; however, the mean I .Q,. 
scores of the experimental group in Replication two were lowe»' than the 
1 .0,. scores of the other three groups. It is interesting to note, 
however, that the mean i .d. of that group was 96.15- Since the mean 
I .d. fell In the middle range it appeared to make little difference. 



TABLE 16 





l.d. MEANS 


FOR ALL 


GROUPS 




Grade 


Repl i cat ion 
Experimental 


1 

Control 


Repl i cat ion 
Exper imenta 1 


2 

Control 


1 


106.08 


116.11 


1 16.29 


91-55 


2 


108.33 


99-48 


106.04 


98.74 


3 


119-07 


84.62 


91.91 


116.89 


4 


103-68 


98.56 


89.81 


101 .67 


5 


102.73 


109-10 


105.76 


106.52 


6 


106.55 


107-50 


96.15 


102.32 
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Summary of the ANCOVA Datai for Each Grade . A visual summary of 
the results of the analysis of covariance for each grade level is 
presented in Table 17 . This table shows that the mean scores of the 
experimental group were significantly higher on the Critical Reading 
Test Total than those of the control group at every grade level. How- 
ever, the differences were at a marginal level of significance in 
grades two and six. 

When the sections of the critical reading test were used as the 
criterion measures, Logic was the only one on which the experimental 
group scored consistently higher than the control group. The mean 
scores of the experimental group on this section were significantly 
higher at all grade levels with the exception of grade four where the 
difference was at a marginal level of significance. In contrast, there 
were no differences on the General section of the test in grades two, 
four, five, and six. In grade one, the control group scored higher 
than the experimental group on this section while in grade three the 
experimental group scored higher. On the Literature section, there 
were again no differences between the treatment groups in grades two, 
four, five, and six. The experimental group did better than the control 
group on this section both in grades one and three; however, the 
difference was at a marginal level of significance in grade one. 

On the General Reading Test Total, there were no differences 
between treatment groups except in grade one where the experimental 
group had significantly higher mean scores than the control group. 

Thus the analysis of covariance data show that children who 
received instruction in critical reading did better on the Critical 
Reading Test Total and the Logic section than children who had not 
received such instruction. At the same time, they did as well as the 
children in the control group on the general reading test at most 
grade levels and better than the control group in the first grade. 

Several differences between replications were found for the 
critical reading and general read i ng tests . These differences occurred 
on the Critical Reading Total in grades four and five; on the Logic 
section in grades one, two and four; on the General section In grades 
three and five; and on the General Reading Test Total in grades one, two, 
four and five. Only on the Literature section did no replication 
differences occur. 

There were also a number of replication by treatment interactions. 
These interactions occurred on every criterion .neasure in grades one 
and three; on the Critical Reading Total, Logic and General Reading Total 
in grade four; and on the Logic and General sections in grade five. 

No Interactions occurred in grades two and six. 

The replication differences and the interactions show that there 
was some factor other than the treatment affecting the differences 
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TABLE 17 

A VISUAL SUMMARY OF DIFFERENCES BETWEEN REPLICATIONS AND BETWEEN TREATMENTS 
AT EACH GRADE LEVEL ON GENERAL AND CRITICAL READING TESTS 
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aSignificant at the .10 level 
*Significant at the .05 level 
**Signif leant at the .01 level 



between groups. An analysis indicates that I ,Q., differences caused 
many of the interactions. Those groups with I .Q,. scores significantly 
higher than the other groups at a particular grade level tended to make 
higher scores on the criterion measures while those groups with I .Q,. 
scores significantly lower than the other groups tended to make lower 
scores. The interactions in grade five and the replication effects 
remain unexplained, but may be due to differences in teacher ability 
on certain critical reading skills such as the use of logical reasoning. 



Differences Across Grade Levels 



in order to make comparisons across grade levels, analysis of 
covariance tests were run on the scores of the criterion measures of 
general reading and critical reading wherever possible. The analysis 
was limited to a comparison of those scores where subjects from different 
grade levels had taken the same forms of each test. Thus comparisons 
were made of first and second-grade subjects on the general reading 
test only, second and third-grade subjects on the critical reading 
test only, and fourth, fifth, and sixth-grade subjects on both the 
general reading and critical reading tests. The analysis of covariance 
design involved the factors of replication, grade, and treatment. 

Grades One-Two . Table 19 presents the F's from the analysis of 
covariance of the scores on the general reading tests for subjects in 
grades one and two and Table 18 presents the main effect means. It 
may be observed from these tables that the subjects in grade two did 
significantly better than those in grade one on the General Reading 
Test Total and that subjects in the experimental group did significantly 
better than those in the control group. 

Although there was no difference between replications, all inter- 
action effects were significant. The replication by grade interaction 
was due to the high scores of grade one subjects in Replication two 
and of grade two subjects in Replication one. There was a replication 
by treatment interaction because subjects in the experimental group 
did better in Replication two than in Replication one while the converse 
was true of subjects in the control group. The grade by treatment 
interaction was due to the fact that the experimental group did better 
than the control group in grade one while there was no significant 
difference between treatment groups in grade two. The interaction of 
replication by grade by treatment was due to a combination of the above 

effects . 

Grades Two- Three . The analysis of scores of subjects in grades 
two and three (see Tables 20 and 21 ) on the Critical Reading Total 
shows no main effect of either replication or grade. However, the 
mean score of the experimental group on the Critical Reading Total was 
significantly higher than that of the control group. The significant 
replicd>tion by treatment interaction is due to the fact that subjects 
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TABLE 18 



UNADJUSTED MEANS ON THE GENERAL READING TEST TOTAL BY REPLICATION, 
TREATMENT, AND GRADE FOR GRADES ONE-TWO 





Repl i cat ion 
R] R2 


Treatment 
T, T2 


Grade 
G] G2 


GENERAL READING 
Pretest 
Post test 
Gain 


TEST TOTAL 

57.W 

73.87 

16.43 


57.23 

73.54 

16.31 


59.79 

76.34 

16.55 


55.22 

71.43 

16.21 


45.62 

63.99 

18.37 


67.44 

82.55 

15.11 



TABLE 19 

F TESTS FROM ANALYSIS OF COVARIANCE FOR TABLE 



GENERAL READING TEST 



Source Total 



Replication 

Gr^e 17..83’V* 

Treatment 5-39* 

Rep. X Grade 6.92** 

Rep. X Treat. 62. }4** 

Grade x Treat. 17.36** 

Rep. X Grade x Treat. 32.58** 



*Sign if leant at the .05 level 

**Sign if leant at the .01 level 



